Generalized Second-Order Value Iteration in Markov Decision Processes

Value iteration is a fixed point technique utilized to obtain the optimal value function and policy in discounted reward Markov decision process (MDP). Here, contraction operator constructed applied repeatedly arrive at solution. first-order method and, therefore, it may take large number of iterations converge Successive relaxation popular that can be solve equation. It has been shown literature under special structure MDP, successive overrelaxation computes faster than standard iteration. In this article, we propose second-order procedure obtained by applying Newton–Raphson scheme. We prove global convergence our algorithm solution asymptotically show convergence. Through experiments, demonstrate effectiveness proposed approach.